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Abstract 



This paper proposes a new algorithm for solving constrained global optimiza- 
tion problems where both the objective function and constraints are one-dimensional 
r^^ • non-differentiable multiextremal Lipschitz functions. Multiextremal constraints 

^ ' can lead to complex feasible regions being collections of isolated points and inter- 

vals having positive lengths. The case is considered where the order the constraints 
are evaluated is fixed by the nature of the problem and a constraint / is defined only 
over the set where the constraint z — 1 is satisfied. The objective function is defined 
only over the set where all the constraints are satisfied. In contrast to traditional ap- 
^ ' proaches, the new algorithm does not use any additional parameter or variable. All 

0^ , the constraints are not evaluated during every iteration of the algorithm providing 

>0 ■ a significant acceleration of the search. The new algorithm either finds lower and 

^N ' upper bounds for the global optimum or establishes that the problem is infeasible. 

Convergence properties and numerical experiments showing a nice performance 
t~^ ' of the new method in comparison with the penalty approach are given. 
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1 Introduction 

In last decades univariate global optimization problems were studied intensively (see 
II71[T5][l8][T9l|22l|28][3l]|37] |43]) because there exists a large number of real-life 
applications where it is necessary to solve such problems (see f6\ [15] |22l |30] |33] l37l 
l40l "). On the other hand, it is important to study these problems because mathematical 
approaches developed to solve them can be generalized to the multidimensional case 
by numerous schemes (see, for example, one-point based, diagonal, simplicial, space- 
filling curves, and other popular approaches in lfT4l[T6l[r7ll23ll26ll29l[37l ). 

Electrotechnics and electronics are among the fields where one-dimensional global 
optimization methods can be used successfully (see |l8l|9l[T0][TT]|24l[33]@:0|). Let us 
consider, for example, the following so-called 'mask problem' for transmitters. We 
have a transmitter (for instance, that of GSM cellular phones) that in a frequency inter- 
val [a,b] has an amplitude A (x) that should be inside the mask defined by functions l{x) 
and u(x), i.e., it should be l{x) < A{x) < u{x). The mask is defined by international 
rules agreed to avoid interference appearing when amplitude is too high for a given 
frequency and by properties of electronic components used to construct the transmitter. 
Then, it is necessary to find a frequency x* S [a,b] such that the power, p{x), of the 
transmitted signal is maximal. 

It happens often in engineering optimization problems (see |l37lj40l) that if a con- 
strained is not satisfied at a point then many other constraints and the objective function 
are not defined at this point. This situation holds in our mask problem because if for 
a frequency t, if happens that A(^) > u{t,) or A(^) < l{t,) then there is no transmission 
and the function /?(^) is not defined. Since the amplitude can touch the mask both from 
its internal and its external parts, isolated points in the admissible region of p{x) can 
take place. If the maximal power p{x*) is attained at an isolated point x*, then this 
point should be discarded from consideration because it cannot be realized in practice. 
Thus, the solution is acceptable only if it belongs to a finite interval of a certain length. 

This problem can be reformulated in the following general framework of global 
optimization problems considered in this paper It is necessary to find the global mini- 
mizers and the global minimum of a function f{x) subject to constraints gj{x) < 0, 1 < 
/ < m, over an interval [a,b]. The objective function /(x) and constraints g^ (jc) , 1 < / < 
m, are multiextremal non-differentiable 'black-box' Lipschitz functions with a priori 
known Lipschitz constants (to unify the description process the designation gm+i (x) = 
f{x) is used hereinafter). Very often in real-life appUcations the order the constraints 
are evaluated is fixed by the nature of the problem and not all the constraints are de- 
fined over the whole search region [a^b]. The worst case is considered here, i.e., a 
constraint gj+i{x) is defined only over subregions where gj{x) < 0. This means that if 
a constraint is not satisfied at a point, the rest of constraints and the objective function 
are not defined at that point. The sets Qj,l < j <m+l, can be so defined as follows 

Qi^[a,b], Qj+i^{xeQj:gj{x)<Q}, 1 < ; < m, (1) 

Since the constraints are multiextremal, the admissible region Qm+\ and regions Qj, 
1 ^ J < 'w, can be collections of intervals having positive lengths and isolated points. 
Particularly, isolated points appear when one of the constraints touches zero, for exam- 
ple, if gj{x) is the square of some function, then gj{x) < only when gj{x) — 0. To be 
implementable in practice, optimal solutions should have a feasible neighborhood of 
positive length thus, an additional constraint is included in the model: a point x* should 



belong to an admissible interval having length equal to or greater than 5 > - a value 
supplied by the final user The set of all such intervals is designated as Q (of course, 
Q C Qm+i)- Eventually found isolated points and feasible subregions having length 
less than 5 should be excluded from consideration. If the case of infeasible problem 
Q =% holds, it should be also determined. 

We can now state the problem formally. Find the global minimizers x* and the 
corresponding value /* such that 

r^f{x*)^mm{f{x):xeQ^}, (2) 

where the objective function f{x) and constraints gj{x)^ \ <i < m, are multiextremal 
functions satisfying the Lipschitz condition in the form 

\gjix')~gjix')\<Lj\x'-x"\, x'yeQj, l<j<m+l, (3) 

and the constants 

0<L;<oo, l<;<m+l, (4) 

are known (this supposition is classical in global optimization (see lfT5l[T7ll28l ')'). Meth- 
ods working on the basis of this assumption are called 'exact' in literature, methods 
estimating these values are 'practical' . On the one hand, the exact methods serve as 
a basis for studying theoretical properties of practical ones and are used as a unit of 
measure of the speed of practical methods. On the other hand, in certain cases, when 
additional information about the objective function and constraints is available, they 
can be applied directly. 

An example of such a problem is shown in Fig. [T] It has two non-differentiable 
multiextremal constraints gi(x) andg2{x). The corresponding sets 2i = [o,b],Q2, and 
23 are shown. The point c belongs to the sets Qi,Q2, and Q^ but c ^ Q . The set Q is 
shown by the grey color. It can be seen from Fig.[T]that the sets 22, 23, and Q consist 
of disjoint subregions and 22,23 contain also an isolated point. 

It is not easy to find a traditional algorithm for solving the problem (|2|l-(|4|l. For 
example, the penalty approach requires that f{x) and gi{x) , I <i <in, are defined over 
the whole search interval [a,b]. It seems that missing values can be simply filled in with 
either a big number or the function value at the nearest feasible point. Unfortunately, 
in the context of Lipschitz algorithms, incorporating such ideas can lead to infinitely 
high Lipschitz constants, causing degeneration of the methods and non-applicability of 
the penalty approach. 

A promising approach called the index scheme has been proposed in f38l (see also 
||39ll40l ) in combination with stochastic Bayesian algorithms. An important advantage 
of the index scheme is that it does not introduce additional variables and/or parameters 
by opposition to classical approaches in ||2l E] [l6l [iTl |25l . It has been recently shown 
in [35 J that the index scheme can be also successfully used in combination with the 
Branch-and-Bound approach. Unfortunately, this scheme can not be applied directly 
for solving the problem ©-(111) because it has good convergence properties when all 
the sets Qj,l < j < m + l, have no isolated points - requirement hardly verified in 
practice without some additional information about the problem. 

Thus, isolated points give serious problems when one has only Lipschitz informa- 
tion. First, because it is not possible to say a priori whether the feasible region has 
isolated points or not (for example, the method from Il35l converges only to global 
minimizers if it is ensured absence of isolated points). Second, in Lipschitz global 
optimization isolated points can lead to two problems: accumulation of trial points in 



Figure 1 : An example of the problem (|2]i-(|4|i. 
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their neighborhood (this happens even if there exists a sub-region where a constraint 
does not touch zero but is only close to zero) and increasing the estimates of Lipschitz 
constants to infinity. This fact means that the search region will be covered by a uni- 
form mesh of trials) if the Lipschitz constant is estimated or the method simply will not 
work if the Lipschitz constant is given - our case - because local adaptively obtained 
information will contradict the given one. Therefore, traditional Lipschitz methods 
cannot be used in the presence of isolated points and, since their absence can be hardly 
determined in practice, developments of methods that are able to work independently 
of the presence or absence of them becomes very important. 

In this paper, such a method is proposed. It evolves the idea of separate consid- 
eration of each constraint introduced in [381 in a new way and reduces the original 
constrained problem to a new continuous problem. The method from |[35l is used as a 
basis for construction of the new scheme. Instead of discontinuous support functions 
proposed in 1351 , new continuous functions are built. These new structures are very 
important because by using them it becomes possible to apply numerous tools devel- 
oped in Lipschitz unconstrained optimization to a very general class of constrained 
problems. It is also necessary to emphasize that the new approach does not introduce 
additional variables and/or parameters during this passage from initial discontinuous 
constrained partially defined problem to the continuous unconstrained one. 

To conclude this introduction it is necessary to emphasize once again that the prob- 
lem of multi-dimensional extensions of one-dimensional Lipschitz global optimization 
methods to many dimensions is a non-trivial serious problem (P. Hansen and B. Jau- 
mard write in their survey on Lipschitz optimization |15| published in the Handbook 
of Global Optimization: 'Large problems (with 10 variables or more) appear to be of- 



ten intractable, at least if high precision is required') and is beyond the scope of this 
paper dedicated to the univariate algorithms and univariate applications. However, the 
approach proposed here is very promising from this point of view. In the future, a num- 
ber of various multi-dimensional extensions (starting from the adaptive diagonal and 
space-filling curves approaches (see Il34ll40l )) of the algorithm presented in this paper 
will be studied. 

The rest of the paper is organized as follows. The new method is described in 
Section 2. Section 3 contains computational results and a brief conclusion. 

2 Continuous index functions and the new algorithm 

The index scheme (see If38ll39l l40l) considers constraints one at a time at every point 
where it has been decided to calculate f{x) determining the index v = v(x), 1 < V < 
m + 1 , by the following conditions 

^>W<0, l<;<v-l, ^vW>0, (5) 

where for v = m + 1 the last inequality is omitted. The term trial used hereinafter 
means determining the index v{x) at a point x by evaluation g,(x), 1 < / < v(x). The 
index v(x) and the value gv(x) (^) ^re called results of the trial. 

The discontinuous index function J{x)^x 6 [a^b], can be written for the problem 
©-dl following 133 

jf \ ^ \ / 0, v(.x) <m+l, 

where the value /* is the unknown solution to this problem. 

Let us start our theoretical consideration by noticing that the global minimizer of 
the original constrained problem ©-(Jill in the case 2 7^ coincides with the solution 
to the following unconstrained discontinuous problem 

J{x*)^mm{J{x):x(^Q^}, (7) 

where 

Q^ = [aM\{Qr,.+ x\Q^}- (8) 

Suppose now that trials have been executed in a way at some points 

a ^ XQ < x\ < . . . < Xi < . . . < Xk = b (9) 

and v, — v(ji:,),0 < i < k, are their starting indexes. Note that the notion of index is 
different with respect to f 38l [39l l40l where the index is calculated once and then used 
in the course of optimization. In this paper, formula ^ defines the starting value for 
the index that can then be changed during the work of the algorithm. 

The points from (|9]l form the list (called hereinafter History List H{k)) of intervals 
[^i7''(],l <i<k, where 

h < ri, I <i<k, 

ri ==/,+!, 1 < /< ^. 

The record x 6 H{k) means that the point x = /, or x = r, for an interval / from H{k). 
Every element /, 1 < / < A:, of the list contains the following information: 

%ri], v{li), V(r,-), gviii)ili), 8vin)in)- (10) 



■/''W=^vmW-| 



The second list, W{k), called Working List is built during the work of the method to be 
introduced by excluding from H{k) intervals where global minimizers of the problem 
©-(Hll can not be located (initially it is stated W (k) = H{k)). In contrast to H{k) where 
the information (fTOl l is calculated once and then is kept during the search, indexes v(/,) 
and v(r,) in W{k) can be changed in the course of optimization. 

In order to pass from the problem ©-(HJi to the problem Q it is necessary to 
estimate the value /* from (|2|i and the set Q . Using the results of trials at the points 
from the row (|9]l the value 

Zl=mm{g,„+i{x):v{x)=m+\,xeW{k)}. (11) 

estimating /* can be calculated if there exist points x with the index v(x) ~m+l. This 
value allows us to define the function j''{x),x £ [a,b],hy replacing the unknown value 
f*m©hyZ*^: 

0, V(x)<m+1, 

Z*, v(x)=m+l. ^'^' 

The following Lemma establishes some useful properties of the functions J{x) and 
/(x). 

Lemma 1 The following assertions hold for the functions J{x) and J (x): 
i. for all points x having indexes v{x) < m+l, it follows J (x) = J{x) > 0; 

it. 

J''ix)<0, xe{x:g,„+i{x)<Zl}. (13) 

Hi. ifv {x) ^ m+l and Z^ > /* then 

j''{x)<J{x). (14) 

Proof. Truth of assertions i - iii follows from definitions of the functions J{x) and 
J\x). ■ 

Particularly, it follows from Lemma[T]that (fl4] l holds if the trial point corresponding 
to Z^ belongs to Q . The estimate (fT4l i is not true if Z^ < /*, the situation which can 
occur only if x| belongs to the set Qm+i \ Q where 

xl = argmin{g,„+i(x) : v(x) ^m+\,xe W{k)}. (15) 

2 

Let us introduce the following continuous index function C(x),x e Q , and study 
its properties. 

C(x) =max{7(x), J{y)-K^r) \x~y\)}, (16) 

where ^v(y) such that Z^(,.) < ^v(.y) < °° is an overestimate of the Lipschitz constant 
corresponding to the function gy[y){y) and J{x) is the discontinuous index function 
from (|6]l. 

As an illustration, the function C{x) corresponding to the problem presented in 
Fig. [T]is shown in Fig. |2] The parts of the function C{x) corresponding to x and y such 
that 

J{x) <J{y)-Ky(^y) \x-y\ 

are shown by the thin line. 



Figure 2: The function C{x) corresponding to the problem presented in Fig.[T] 




If Q° ^ 0, the global minimizers of the original constrained problem (|2li-(|4]l coin- 



cide with the solutions of the following continuous problem 

C{x*)^mm{C{x):xeQ^}. 



(17) 



In the case g — the set 2 —[a,b]\ 2m+i and we have v(x) < m-l- l,x G 2 .Thus, 
due to Lemma[Tl it follows 

C(x)>0, xeQ^ (18) 

Similarly to definition of the function j''{x), the value Z^ is used to define the 
functions C*(x),x £ [a,b], as follows. 

C'^W^ max{/W, /(y)-/:v(,) \x-y\)}. (19) 

ye[a,b] 

Lemma 2 The following assertions hold for the functions C{x) and C {x): 

i. inequalities C{x) > J{x), C (x) > J (x) hold over the set Q ; 

a. if v(x) < m + 1 then C*^(x) > 0; 

Hi. if v{x) = m+ I and X G W{k) then C (x) > 0; 

iv. if X £ {x: gm+i (x) < Z^} then C (x) < 0. 

V. if X* e Q^ then C''{x) < C{x), x e Q^. 

Proof. The truth of the assertions follows from Lemma[T]and formulae (fTTTi.lfTSll. ( fTSI l, 
and (O. ■ 

It follows from Lemma|2]that if x^, e Q , the global minimizers cannot be located 
in zones where C'^{x) > 0,x £ Q . Over every interval [/,-,r,] we are interested in sub- 
regions having the index greater or equal to 

vy = max{v(/,),v(r,)}, 



because, due to construction of the function C*^(x), only these subregions can probably 
contain a global minimizer. It can be shown (see 1351 ) that 



[y7,ytl v(/,-)=v(r,), 
[li,n]nQ^c{ [y;,nl v(/;)<v(n), (20) 

[li,yt], v(Z,)>v(r,), 

where 

yr = '<+za-)/^v(/,), (21) 

yt^n-z{r,)/K^(^,_), (22) 

and z{x) =J^{x). Let us call any value 7?,- , 1 <i<k, characteristic of the interval [/,■ , r,] 
if the following inequality is true 

min{C''(x) : x G [li,ri],v{x) = vj} > Rt. (23) 

It follows from assertion i of Lemma|2]and ll35l that ( [23T l is fulfilled for /?, — Ri where 

r 0.5{z{l,)+z{ri)~K^^,_){ri-li)), v(Z,-) = v(r,), 

Ri^R{li,ri)^\ z{r,)-K^(,^)(ri~yr), v(/0 < v(r,0, (24) 

I z{li)-K^(i,){yi-U), v(ZO>v(r,). 

The characteristic Ri from (l24l l depends only on the values of the function j'^{x) evalu- 
ated at the points /, and r,-. It does not use any information from other intervals belong- 
ing to the working list W(A:). 

We are ready now to introduce the Algorithm working with Continuous Index Func- 
tions (ACIF). It either solves the problem ( [TtI i or determines that the case ( fTSl l takes 
place. The ACIF works by calculating characteristics /?, initially using (l24l i and then 
improving them during the search by constructing the function C*(x). On the one 
hand, the method tries to find a good estimate Z^. On the other hand, it searches and 
eliminates from 'W{k) intervals that cannot contain x* using the fact following from 
Lemma|2]and ( |23] | that if x^ e Q , an interval [Ij^rj] having a characteristic Rj > can 
be eliminated from consideration. The constraint introducing the parameter 5 helps to 
exclude more intervals. 

Let us take a generic interval [/?,/",], 1 < f < ^(^ + 1), from the working list and 
calculate its characteristic R{lt,rt). We will also show how the function C'^(x) allows 
us to improve characteristics of intervals adjacent to [Zf,r,]. 

Initially characteristic for the interval [lt,rt] is calculated as R{lt,rt) — R{lt,rt). If 
R{ltTrt) < or v(/,) = v(r,), then the characteristic R{lt,rt) has been computed. If 
R{lt,rt) > and v(/r) < v{rt) go to the operation Backward motion. Otherwise execute 
the operation Onward motion. 

Backward motion. Exclude from W{k+l) all the intervals / such that 

zirr)~K^^,-,)ir,~li))>0, l<j+l<i<t-l, (25) 

where the interval j violates (|25] |. Calculate the value 



where 



0.5(z(Z,)+z-(o)-^v(.,)(o -';)), V(/,) =v(r,) 

RJ = '{ z-{rj)~K^(^,.){rj~lj~z{lj)/K^,^i^)), v{lj) <v{r,) (26) 

zih)-Kv{ij)irj~lj~Z-{rj)/K^l,,)), v(/y) >v(r,) 

z-{rj)=z{n)-K,^r,)irr-rj)). (27) 



Figure 3: Improving characteristics by the operation Backward motion. 




If Rj < Rj, set in the working list W{k+l) 

z{rj)^z-{rj), v(o)=v(r,), Rj=RJ, 

maintaining in the history list //(A: + 1) the original values of g^/^.Urj) and v(ry). Cal- 
culate the number ^(A; + 1 ) of the intervals in W{k+ I). 

An illustration to the operation Backward motion is given in Fig. [3] Three intervals 
are presented in Fig. [3) 

[li-2,ri-2]^[p,q], [li-i,ri^i] = [q,h], [li,n] ^ [h,d]. 

Suppose that the function C''{x) has been evaluated at the points p,q,h, and d and 

v(/')-v(^)=v(/;)<v(£/). 

Characteristic ^,_2 of the interval [p,q] is negative and the bold line shows the zone 
where the global minimizer could be probably found. The same situation holds for 
the interval [q,h]. Since the characteristic Rj of the interval [h,d] is positive and 
v(/i) < v{d), the operation Backward motion starts to work. It can be seen from Fig. [3] 
that the new characteristics /?,_2 and /?,_i calculated using information obtained at the 
point d are positive and, therefore, the intervals [p,q] and [q,h] cannot contain global 
minimizers and can be so excluded from the working list. 

Onward motion. Exclude from W{k+l) all the intervals i such that 

Z{lr)-K^il,){n-Ir))>0, t+l<i<j-l<q{k), (28) 

where the interval j violates ( |28] ). Calculate the value 

r 0.5{z+{lj)+z{rj)-K,^,.^{rj-lj)), v{lt)=v{rj) 

Rj={ zirj)-K^^,^){rj-lj^z+ilj)/K^ii,)), v(Z,)<v(o) (29) 

[ z+{Ij)-K^i,,){rj~lj~z{rj)/K,^,^)), v(Z,)>v(o) 



where 

z+{lj)=zih)-K,^,,){lj-lr)). (30) 

If /?+ < Rj, set in the working Hst W{k+l) 

z{lj)^z+{lj), v(/,-)=v(Z,), Rj = RJ, 

maintaining in the history Ust H{k+ 1) the original values of g^/i.\{lj) and v{lj)- Cal- 
culate the number q{k+ I) of the intervals in W (A: + 1 ) . 

In order to describe the method we need some definitions and initial settings. It is 
supposed that: 

- the search accuracy < e < 5 has been chosen, where 5 is from Q; 

- two initial trials have been executed at the points x^ — a and x' — b; 

- It has been assigned W(l) =//(!) = [jc^jx'] and the number t of the interval to be 

subdivided at the next iteration has been set to f = 1 ; 

- the values Z| and 

M* = max{v(x') : < / < ;t} (31) 

have been calculated for k= 1; 

- the set V containing the points x ^H(k) such that x e Qm+\ \ Q has been set to 

Suppose now that k,k> 1, iterations have been made by the ACIF, the list H{k) 
contains k intervals, W{k) contains q{k) intervals for which characteristics have been 
evaluated, and an interval [It , r,] for subdivision has been found. The choice of the next 
interval to be subdivided is made as follows. 

Step 1. (Subdivision and the new trial.) Update W{k+ 1) and H{k+ 1) by 
substituting the interval [Zf,r,] in W{k) and H{k) by the new intervals [/,,x*^+'], 
[;c*+^,rf] where 

0.5{yr+y+), v(h)=v(r,) 

x'+' = { 0.5{y7+rt), v{lr) <v{n) (32) 

0.5^+3'+), va)>v(r,) 



Execute the {k+1 )-th trial at the point .x*^+' and, as the result, obtain the values 
v(x*^+') and g^,k+i\{x''^^)- Recalculate M'^+^ 

Step 2. (Calculation of the estimate Z^^^ and characteristics.) Associate 
with the point x''^ ' the value z*^+ ^ — 7*^+ ' (x'^^ ' ) and recalculate the estimate Z^^ j 
if v(x'^+^) =m + l. lfZ*^^i < Zl then for all points xeW{k+\),x^ x''+^ , such 
thatv(x) =m+l setz(x) =z(x)+Z^— Z^^j and recalculate characterisitcs of the 
intervals in W(A;+ 1). Otherwise calculate characteristics only for the intervals 
[l,,x''+^] and [x''+\rt]. Go to Step 3. 

Step 3. (Finding an interval for the next subdivision.) If W(A:+ 1) =0, then 
Stop (the feasible region is empty). Otherwise, find in the working list W{k+l) 
an interval [lt,rt] such that 

t = min{argmin{/?,' : 1 < i <qik+l)}} (33) 

and go to Step 4. 
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Step 4. (Verifying appurtenance to the set Q .) If the interval to be subdi- 
vided can belong to the set Q then go to Step 5. Otherwise exclude all found 
intervals that are out of Q from the working list and include the points forming 
these intervals and having the index m + 1 in the set V . If the point jc^^j belongs 
to one of the excluded intervals then go to Step 6 otherwise go to Step 3. 

Step 5. (Verifying accuracy.) If the inequality 

r,-/, >e (34) 

holds, then go to Step 1 . In the opposite case. Stop (the required accuracy has 
been reached). 

Step 6. (Restarting.) Recalculate the estimate Z^^j without usage of the 
points included in V . Form the new set W(A;+ 1) including in it all the inter- 
vals from H{k+\) that do not contain points from V and intervals containing 
points X S y such that z{x) > Z^^j. For all intervals in W{k + 1) recalculate 
characteristics Ri applying backward motion for all intervals / having Rj > if 
v(/r) < v(r;) and onward motion if v(Z,) > v(r,). In the latter case, characteris- 
tics of the intervals satisfying (|28] | are not calculated. Exclude from W{k+l) all 
the intervals having positive characteristics. Then go to Step 4. 

Step 4 executes an important operation - verifying appurtenance to the set Q . 
To do this we check whether the interval [lt,r,] chosen for subdivision can contain a 
feasible interval having a length greater than 5. Four cases should be considered. 

i. (Casev{lt) <m+l,v{r,) <m + l.) If 

yt -yi ^n-k- z{r,)/K^^,,) -z{l,)/K^(i,) < 5 (35) 

then [lr,r,] ^ Q because over [lt,rt] only the interval [yrjyi^] can possibly con- 
tain a global minimizer but its length is less than 5. 

ii. (Case v{lt) — m + 1, v(r,) <m+ I.) Analogously, if 

r,-l,-z{r,)/K^(^^^ >8 

then the interval [/r,r,] can belong to the set Q . Otherwise, if in the history list 
H{k+\) there exists an interval [lj,rj\J < t, such that v(/j) <m+l and 

r, - 1 J - z(r,)/^v(r,) - z{lj)lK^{i,) < § (36) 

or v(Z;) =m+l,j= 1 , and 

rr~lj~z{n)/K^^„)<?, (37) 

then all the intervals [Ij , r^] ,...,[/,, r,] ^ Q^ and the corresponding points rj,...,lt & 

Q,n+l\Q^ 

iii. (Case v{lt) < m + I, v(r,) = m + I.) This case is considered analogously 
to the previous one but confirmation of possibility for [/f,r,] to belong to Q is 
searched among intervals / > t. 

iv. (Case V (It) —v{rt) —m+l.) This case is a combination of the cases ii and iii. 
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The introduced procedure verifies inclusion [lt,r,] S Q for all possible combina- 
tions of indexes v(/(), v(r,). Of course, it is also possible to simplify Step 4 and verify 
only condition (l35l l - the rule determining during the search the major part of intervals 
belonging to Qm+i \ Q ■ In this case, after satisfying the stopping rule from Step 5, 
it is necessary to check whether the found solution jc^^j belongs to Q,„+i \ Q and, if 
necessary, to reiterate the method starting from Step 6. 

The following situations can, therefore, hold after fulfillment of the stopping rule: 

i. The algorithm has finished its work and the working list is empty, then Q =& 
and the set V contains the points from Qm+i\Q if any. 

ii. The working list is not empty and it does not contain intervals [lp,rp] such that 
Rp<0 and 

max{v(Zp),v(rp)}<m+l. (38) 

In this case it is necessary to check locally in the neighborhood of x*f, whether 
xl £ Q ■ If this situation holds, then the global minimum z* of the problem 
©-(m can be bounded as follows 

z*e[Rr^k)+zlzl] 

where 7?,(^) is the characteristic corresponding to the interval number t = t{k) 
from ( l33T l. In the opposite case it is necessary to include the point x^ in V and 
to return to Step 6. 

iii. The last case considers the situation where the working list is not empty and there 
exists an interval [IpTrp] such \haXRp < and ( l38T l holds. Again, it is necessary 
to check locally in the neighborhood of xl whether xl G Q . If this analysis 
shows that x^ ^ Q then it is necessary to include the point x^ in V and return 
to Step 6. Otherwise, the value Z^ can be taken as an upper bound of the global 
minimum z* ■ A lower bound can be calculated easily by taking from the working 
list the trial points x,- such that v(x,) —m+l and constructing for f{x) the support 
function of the type ll28l using only these points. The global minimum of this 
support function over the intervals belonging to the working list will be a lower 
bound for z* ■ 

Consider now the infinite trial sequence {x''} generated by the algorithm ACIF 
when e = in the stopping rule (|34] |. We denote by X* the set of the global minimizers 
of the problem ©-dill and by X' the set of limit points of the sequence {x*^}. The 
following two theorems describe convergence conditions of the ACIF. Since they can 
be derived as a particular case of general convergence studies given in ifTTl (Branch- 
and-Bound approach) and f32l (Divide the Best algorithms) their proofs are omitted. 

Theorem 1 If the problem ^-& is feasible, i.e. Q^ ^ 0, thenX* ^X'. 

Theorem 2 // the problem (O-fHl is infeasible then the algorithm ACIF stops in a 
finite number of iterations. 



3 Numerical comparison and conclusion 

The ACIF has been numerically compared to the algorithm (indicated hereinafter as 
PEN) proposed by Pijavskii (see ll28l [T5J ) combined with a penalty function. The 



12 



Table 1; Numerical results obtained by the PEN on 10 non-differentiable and 10 dif- 
ferentiable problems. 



Problem 


Non-differentiable 


Differentiable 




Iterations 


Evaluations 


Iterations 


Evaluations 


1 


247 


494 


83 


166 


2 


241 


482 


954 


1906 


3 


797 


1594 


119 


238 


4 


272 


819 


1762 


5286 


5 


671 


2013 


765 


2295 


6 


909 


2727 


477 


1431 


7 


199 


597 


917 


2751 


8 


365 


1460 


821 


3284 


9 


1183 


4732 


262 


1048 


10 


135 


540 


2019 


8076 


Average 


501.9 


1545.8 


817.9 


2648.1 



PEN has been chosen for comparison because the method of Pijavskii in literature (see 
lfT4l [161 [TtI l23l l26l l29l l37l ) is used as a kind of the unit of measure of efficiency 
of the new Lipschitz global optimization algorithms and it uses in its work the same 
information about the problem as the ACIF - the Lipschitz constants for the objective 
function and constraints. The usage of the penalty scheme allows us to emphasize 
advantages of the index approach. 

Since the PEN in every iteration evaluates the objective function f{x) and all the 
constraints, twenty feasible test problems (ten differentiable and ten non-differentiable) 
introduced in ifTJl have been used for testing the new algorithm. The ACIF has also 
been applied to one differentiable and one non-differentiable infeasible test problems 
from lfT3l . In all the experiments there has been chosen the original (see ifTSi ') order 
the constraints are evaluated during optimization, without determining the best for the 
ACIF order. 

In the PEN, the constrained problems were reduced to the unconstrained ones as 
follows 

P*(x) =/(x)+Pmaxtei(x),g2W,...,§N„W,0} (39) 

and coefficients P from lfT3l have been used. The same accuracy e = \Q^'^[b — a) 
(where b and a are from (O) and the starting trial points a and b have been used in all 
the experiments for both ACIF and PEN. 

Table[T]contains numerical results obtained for the PEN. The column "Evaluation" 
shows the total number of evaluations equal to 

where A^,, is the number of constraints and Niter is the number of iterations for each 
problem. 

Tables 12] and |3]present numerical results for the new method for 5 = e and 5 = lOe. 
The columns in the Tables have the following meaning: 

- the columns Ng^ , A'^,, and Ng^ present the number of trials where the constraint 
^(,1 < ' < 3, was the last evaluated constraint; 

- the column Nf shows how many times the objective function /(x) has been eval- 
uated; 
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Table 2: Results obtained by the new algorithm on the non-differentiable problems. 



Problem 



A'., 



N„. 



N„. 



Nf 



Iter. Eval. 



Ne 



No. 



N„. 



10£ 

Nf 



Iter. Eval. 



Average 



79 

50 

131 

387 
401 

1839 
205 
143 

1049 
141 
53.2 33.0 23.0 102.3 185.5 442.5 



1 


23 


- 


- 


28 


51 


2 


18 


- 


- 


16 


34 


3 


95 


- 


- 


18 


113 


4 


107 


14 


- 


84 


205 


5 


153 


88 


- 


24 


265 


6 


16 


16 


- 


597 


629 


7 


52 


18 


- 


39 


109 


8 


28 


11 


3 


21 


63 


9 


8 


81 


49 


183 


321 


10 


32 


3 


17 


13 


65 



23 


- 


- 


28 


51 


79 


17 


- 


- 


16 


33 


49 


80 


- 


- 


18 


98 


116 


82 


11 


- 


84 


177 


356 


14 


66 


- 


24 


204 


318 


16 


15 


- 


597 


628 


1837 


49 


14 


- 


39 


102 


194 


28 


11 


3 


21 


63 


143 


8 


59 


32 


183 


282 


954 


30 


2 


17 


13 


62 


137 



44.7 25.4 17.3 102.3 170.0 418.3 



Table 3: Results obtained by the new algorithm on the differentiable problems. 



Problem 



N„ 



N„. 



N„. 



Nf 



Iter. Eval. 



A'. 



A',,. 



N. 



10£ 

Nf 



Iter. Eval. 



1 


10 


- 


- 


13 


23 


36 


10 


- 


- 


13 


23 


36 


2 


199 


- 


- 


21 


220 


241 


155 


- 


- 


21 


176 


197 


3 


40 


- 


- 


22 


62 


84 


38 


- 


- 


22 


60 


82 


4 


480 


127 


- 


189 


796 


1301 


212 


73 


- 


189 


474 


925 


5 


8 


13 


- 


122 


143 


400 


8 


13 


- 


122 


143 


400 


6 


14 


55 


- 


18 


87 


178 


13 


34 


- 


18 


65 


135 


7 


36 


13 


- 


241 


290 


785 


35 


13 


- 


241 


289 


784 


8 


94 


21 


5 


82 


202 


479 


80 


19 


5 


82 


186 


461 


9 


7 


35 


6 


51 


99 


299 


7 


32 


6 


51 


96 


293 


10 


36 


14 


174 


1173 


1397 


5278 


35 


10 


92 


1173 


1310 


5023 


Average 


92.4 


39.7 


61.7 


193.2 


331.9 


908.1 


59.3 


27.7 


34.3 


193.2 


282.2 


833.6 



- the column "Eval." is the total number of evaluations of the objective function 
and the constraints. This quantity is equal to: 

- Ng^ + 2 X Nf, for problems with one constraint; 

- Ng^ + 2 X Ng2 + 3 X Nf, for problems with two constraints; 

- Ngi+2x Ng2 + 3 X Ngj +4xNf, for problems with three constraints. 

It can be seen from the Tables that in all the experiments the ACIF significantly 
outperforms the PEN both in iterations and evaluations. The ACIF works faster if the 
difference between 5 and e increases. This effect is especially notable for problems 
where it is necessary to execute many iterations out of the feasible region (see columns 



Ng2, Ngj for non-differentiable problems 3-5, 9 and differentiable problems 2, 4, 



8, 10). 

Note that the penalty approach requires an accurate tuning of the penalty coeffi- 
cient in contrast to the ACIF that works without necessity to determine any additional 
parameter. Moreover, when the penalty approach is used and a constraint g{x) is de- 
fined only over a subregion [c,d] of the search region [a,b], the problem of extending 
g{x) to the whole region [a,b] arises. The ACIF does not have this difficulty because 
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the constraints and the objective function are evaluated only within their regions of 
definition. 

Finally, the penalty approach is not able to determine whether a problem is in- 
feasible. The ACIF with 5 = e has determined infeasibility of the non-differentiable 
problem from 1 13 1 in 86 iterations consisting of 81 evaluations of the first constraint 
and 5 evaluations of the first and second constraints (i.e., 91 evaluations in total). The 
infeasibility of the differentiable problem from 1 13 1 has been determined by the ACIF 
with 5 = e in 38 iterations consisting of 9 evaluations of the first constraint and 29 
evaluations of the first and second constraints (i.e., 67 evaluations in total). Naturally, 
the objective functions have not been evaluated in both cases. 

In conclusion, we illustrate performance of the new method (see Fig. |4| and the 
PEN (see Fig. |5]) on the non-differentiable problem 9 from 1 1 3 1 . 



subject to 



min fix) = 3 — 2 exp ( — ( x 

xgIoaY ' ^V 2 V 5 



5 . /ll 
sm I - sm — X 
2 V 5 



22 
sm I 7t I — — X 



gi{x) = 3 exp - 



100" 



-x'-^\<0 



g2{x) = < 



6 X 



1 



- \ X 

4 \ 2 



1 



^aW 



4 
5 



24 



x<i 

x>i 

6 

+ 25 



<0, 



X 

20 



<0. 



The problem has 3 disjoint feasible subregions shown in Fig. |4]by continuous bold in- 
tervals on the line f{x) — 0, the global optimum is located at the point.^c* = 0.95019236 
(see Fig. |4|i. The objective function is shown by a solid line and the constraints are 
drawn by dotted/mix-dotted lines. 

The first line (from up to down) of "H-" located under the graph of the problem 9 
in the upper subplot of Fig. |4] represents the points where the first constraint has not 
been satisfied (number of iterations equal to 8). Thus, due to the decision rules of 
the ACIF, the second constraint has not been evaluated at these points. The second 
line of "H-" represents the points where the first constraint has been satisfied but the 
second constraint has been not (number of iterations equal to 59). In these points both 
constraints have been evaluated but the objective function has been not. The third line 
of "H-" represents the points where both the first and the second constraints have been 
satisfied but the third constraint has been not (number of iterations equal to 32). The last 
line represents the points where all the constraints have been satisfied and, therefore, 
the objective function has been evaluated (number of evaluations equal to 183). The 
total number of evaluations is equal to8 + 59x2 + 32x3 + 183x4 = 954. These 
evaluations have been executed during 8 + 59 + 32 + 183 = 282 iterations. The lower 
subplot in Fig.|4]shows dynamics of the search. 

Fig. |5] shows the penalty function corresponding to P = 15 and dynamics of the 
search executed by the PEN. The line of "H-" located under the graph in the upper 
subplot of Fig. |5] represents the points where the function ( l39T l has been evaluated. 
The number of iterations is equal to 1183 and the number of evaluations is equal to 
1183x4 = 4732. 



15 



Figure 4: Behaviour of the new method on the non-differentiable problem 9 from ifTSl . 
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